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Abstract Body 


Background / Context: 

Description of prior research and its intellectual context. 

Educators, policymakers, and the general public agree that social skills should be taught to 
children (Bushaw & Lopez, 2013; Camilli, Vargas, Ryan, & Barnett, 2010; Dusenbury, Zadrizil, 
Mary, & Weissberg, 2011; HR 1875, 2013; Meier, DiPerna, & Oster, 2006; National Research 
Council, 2012). This support may be a reflection of the growing recognition that social skills are 
critical to both academic and life success (French & Conrad, 2001; Hawkins, Catalano, & Miller, 
1992; Kupersmidt & DeRosier, 2004; Solberg, Olweus, & Endresen, 2007) and that with 
training, these skills can improve significantly (DeRosier, 2004; Durlak, Weissberg, Dymnicki, 
Taylor, & Schellinger, 2011; Zins, Weissberg, Wang, & Walberg, 2004). Social skills training is 
traditionally delivered in person, and logistical barriers such as the need for trained 
administrators, cost, and time and travel requirements limit the reach of traditionally-delivered 
programs (Mueser & Bellack, 2007) and many children who could benefit from social skills 
training do not receive it. 

In an effort to bridge this gap between evidence-based social skills training and populations in 
need, we have developed an Intelligent Social Tutoring System (ISTS) that fosters learning 
through adaptive interaction between the student and the program. As the student interacts with 
the program, the software responds by presenting the most appropriate content, much the same 
way a human tutor would. 

ISTSs offer an innovative way to advance social skills training, address barriers to program 
delivery, and reach children who may not receive traditional social skills programs. By 
presenting simulated situations analogous to those encountered in real life, engaging ISTSs allow 
children to learn and practice social skills in preparation for real world execution. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

The purpose of Study 1 was to determine whether children with social skills deficits would find a 
prototype of the intervention likeable and usable, and whether children who interact with the 
prototype show reduced psychosocial distress and greater behavioral and emotional strength 
compared to children who do not interact with the program. The purpose of Study 2 was to 
replicate and extend the findings of Study 1 to the fully developed program and demonstrate its 
efficacy on a broader set of academic and social skills outcomes. 

Setting: 

Description of the research location. 

In both studies, the intervention was delivered completely online. Children interacted with all 
aspects of the program at home via our secure project webpage. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 
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Study 1 included 36 participants and Study 2 included 144 participants. Participant 
characteristics for each study are shown in Table 1. (Please insert Table 1 here). 


Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

To develop the interactive program, Adventures aboard the S.S. GRIN, we translated the 
evidence-based Social Skills Group Intervention (SSGRIN) curriculum into a game-like 
instructional program by designing in-game tasks that required children to apply specific social 
skills in order to solve each social problem encountered. Developed at 3C Institute and currently 
used with thousands of children across the United States and abroad, the in-person SSGRIN has 
been repeatedly tested resulting in a strong empirical base for its effectiveness (DeRosier, 2004; 
DeRosier & Gilliom, 2007; DeRosier & Marcus, 2005). To create the Adventures aboard the S.S. 
GRIN, an interdisciplinary team of psychologists, artists, and computer programmers 
collaborated to transform the original curriculum into a game-like computer program to actively 
engage children in social problem solving tasks through interactive software technology that 
reflects best practices for children’s educational interactive software (Garris, Ahlers, & Driskell, 
2002; Mathan, & Koedinger, 2002; Roussou, 2005). We used an integrative approach that 
incorporated research-supported social skills training strategies (e.g., behavioral, cognitive, and 
emotional social skills) into instruction to help children think through a given social situation, 
consider various approaches, and evaluate the success of a chosen option toward reaching a 
social goal. Study 1 included four episodes focused on respect, setting and working toward goals, 
impulse control/responsibility, and communication. Study 2 included those episodes as well as 
new episodes that covered perspective taking, social initiation, cooperation/compromise, and 
emotion regulation, as well as a review episode. The level of social problem difficulty increased 
both within and across the episodes. The software monitored children’s progress in real time and 
responded with appropriate content, feedback, and pedagogical assistance based on player 
actions. We incorporated proven training strategies of scaffolding, coaching and reinforcement, 
and practice opportunities into the program. At appropriate times, individualized feedback was 
provided, and when gameplay indicated that a skill was not satisfactorily demonstrated the child 
was provided structured guidance and continued practice opportunities. Achievements were 
provided for outstanding performance. All text-based dialog was accompanied by audio 
voiceover. 

Research Design: 

Description of the research design. 

Program delivery and all assessment was conducted via our secure project website. We 
conducted two studies to test the potential promise of this innovative computerized method for 
social skills training. In Study I, we examined the impact of participation in the first half of the 
program. The program in Study 1 consisted of four instructional episodes; children completed 38 
rooms (i.e., discrete visual spaces), interacted with 42 unique characters, and completed 14 social 
problem-solving scenarios over approximately 90 minutes of gameplay. In Study 2, we expanded 
the investigation to the full nine episodes of the program, which approximately doubled the 
rooms, characters, social problem-solving scenarios, and gameplay time. Our research questions 
were: (1) would children with social skills deficits find the program likeable and useable? and (2) 
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would interaction with the program be associated with improvement in psychosocial, emotional, 
and behavioral outcomes? 


Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

Following interaction with the program, children provided feedback on their experience. Item 
summaries are shown in Table 2. For usability, children answered three questions related to 
program mechanics and their understanding of the program. To assess the degree to which 
children enjoyed their interactions with the program, they answered seven questions about the 
likeability of the program. (Please insert Table 2 here). 

In addition, to measure the effect of the program on psychosocial, emotional, and behavioral 
outcomes, children completed the Youth Outcomes Questionnaire, Second Addition (YOQ-2.01; 
Burlingame et al., 2005) and the Behavioral and Emotional Rating Scale, Second Edition 
(BERS-2; Epstein, 2004). 

Findings / Results: 

Description of the main findings with specific details. 

Study 1 results are presented here. Study 2 data collection will be completed in December 2014; 
we will present Study 2 findings at the conference. Study 2 will include the measures from Study 
1, as well as measures of academic and school functioning and social skills knowledge. 

Usability and Eikeability . We first examined whether children with social skills deficits would 
find the program likeable and usable. These characteristics were important to establish because 
the program’s value as a social skills tutorial would be greatly reduced if children are not able 
and willing to use it independently. The results are shown in Table 2. 

All usability item means were greater than 4.0 on a 5 -point Eikert-type scale, indicating the 
program was understandable and usable to children. Children understood the program in terms of 
gameplay mechanics, the characters’ speech, and how to select actions and responses, indicating 
that gameplay was compatible with transmission of social skills instruction. 

Eikeability item means were also all greater than 4.0 on a 5-point Eikert-type scale, indicating 
high enjoyment of the program. Children gave high ratings for the game and characters, and 
indicated a desire for replay and for more games like this. 

Psychosocial Distress . We examined whether interaction with the program was associated with 
reduced psychosocial distress by examining change scores for the treatment and control groups, 
and by conducting an ANCOVA. Change scores, shown in Table 3, highlighted the practical 
significance of our findings. We calculated the number of children within each treatment group 
who had significant changes in scores (as provided in the Y-OQ 2.01 manual) across each 
subscale and for the total Y-OQ 2.01 psychosocial distress index. “Significant change” is a 
reliable change index used to determine whether the change exhibited by an individual is 
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clinically significant; individual’s scores must change by a minimum amount based on analysis 
of norm data by the measure’s developer to be eonsidered significantly different from pretest to 
post-test. Compared to the control group, children in the treatment group had significant 
improvement across each subscale and on the overall index score. (Please insert Table 3 here). 

We also conducted a one-way ANCOVA, with treatment condition as the independent variable 
and Y-OQ 2.01 pretest score as the covariate. We found that children in the treatment group 
evidenced significantly lower psychosocial distress at post-test (M=23.32, 50=18.79) than 
children in the control group (M=48.71, 50=42.16), when controlling for psychosocial distress at 
pre-test, 0(1, 34) = 4.13, p = .05. 

Behavioral and Emotional Strength . We followed the same statistical procedures as for the Y-OQ 
2.01, first examining change scores and then conducting an ANCOVA. As shown in Table 4, 
more children in the treatment group than in the control group showed improvement across each 
BERS-2 subscale and on the overall index, indicating improved behavioral and emotional 
strength associated with program interaction. (Please insert Table 4 here). 

The one-way ANCOVA, with treatment condition as the independent variable and BERS-2 
pretest score as the covariate, showed that children in the treatment group showed significantly 
higher levels of behavioral and emotional strength at post-test (M=93.21, 50=13.41) than 
children in the control group (M=86.94, 50=19.61), when controlling for psychosocial distress at 
pre-test, 0(1, 34) = 5.35, p < .05. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

The results of Study 1 show strong support for using an online social skills training to improve 
emotional, behavioral, and psychosocial outcomes for children with social skills deficits. We 
expect the findings of Study 2, available by the conference date, to support and expand these 
findings. 


By integrating developmental theory, research, computer science, gaming theory, and appealing 
graphics, we created an engaging and effective interactive social tutoring system for social skills 
training. This methodology has the potential to overcome many of the barriers of traditional 
methods and can deliver services much more broadly than existing methodologies. Online 
programs can be made available to any individual or organization (e.g., schools, mental health 
providers, community/youth groups) and can be used in any setting with a computer. This 
widespread availability means that online social skills training programs can close the gap 
between proven interventions and their use by those most in need, which has important 
implications for fostering optimal development for children at risk for poor social, emotional, 
and academic outcomes. 
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Not included in page count. 
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Appendix B. Tables and Figures 

Not included in page count. 

Table 1. Participant Characteristics 



Study 1 (n=36) 

Study 2 (n=144) 

Age 



% 7/8 years 

37 

36 

% 9/10 years 

30 

46 

% 11/12 years 

33 

15 

Sex 



% female 

27 

42 

% male 

73 

58 

Race 



% white 

63 

46 

% non-white 

37 

54 

BASC-2 scores 



Withdrawal: % Average/ At Risk/Clinical 

23/25/52 

12/35/53 

Aggression: % Average/ At Risk/Clinical 

42/41/17 

47/33/20 
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Table 2. Descriptive Statistics for Usability and Likeability Ratings 



M 

SD 

Usability 



Understood characters’ speech 

4.45 

.23 

Understood how to move and talk 

4.47 

.21 

Understood what Avatar should do next 

4.18 

.13 

Likeability 



Liked program overall 

4.39 

.18 

Thought program was fun to play 

4.46 

.15 

Liked characters 

4.40 

.09 

Liked art and graphics 

4.47 

.07 

Would like to play more programs like this 

4.40 

.14 

Friends would like to play this program 

4.12 

.14 

Would like to play this program again 

4.05 

.11 
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Table 3. Percentage of Participants Demonstrating Significant Changes on the Y-OQ 2.01. 


Y-OQ 2.01 subscale 

Treatment Group 

Control Group 

Interpersonal distress 



Improved 

42% 

35% 

Deteriorated 

0% 

0% 

Somatization 



Improved 

11% 

6% 

Deteriorated 

0% 

0% 

Interpersonal relations 



Improved 

21% 

12% 

Deteriorated 

11% 

0% 

Social problems 



Improved 

0% 

0% 

Deteriorated 

0% 

0% 

Behavioral dysfunction 



Improved 

29% 

12% 

Deteriorated 

0% 

0% 

Critical items 



Improved 

26% 

24% 

Deteriorated 

0% 

6% 

Y-OQ 2.01 Total Score 



Improved 

68% 

52% 

Deteriorated 

0% 

0% 
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Table 4. Percentage of Participants Demonstrating Significant Changes on BERS-2. 


BERS-2 subscale 

Treatment Group 

Control Group 

Interpersonal distress 



Improved 

26% 

24% 

Deteriorated 

0% 

12% 

Eamily involvement 



Improved 

16% 

12% 

Deteriorated 

0% 

12% 

Intrapersonal strength 



Improved 

26% 

0% 

Deteriorated 

0% 

0% 

School functioning 



Improved 

26% 

12% 

Deteriorated 

26% 

18% 

Affective strength 



Improved 

42% 

12% 

Deteriorated 

0% 

6% 

Total strength index 



Improved 

16% 

12% 

Deteriorated 

0% 

18% 
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